Feature Engineering Progress
Outcome Variable Re-Transformation
As noted in the previous memo, we originally planned to apply a log10 tranformation to our outcome variable, an index measuring how queer a ZCTA region was based on parameters defined in this dataset from data world1. However, the appearance of success from the log transformation was based off removal of 0 values, which got transformed to negative infinity. Examining the original data more closely, we find the following things:
- Calculations for
totindex(the index measure) are not always easily decipherable, and for a few ZCTAs seem to be wrong entirely - Some values are 0 when some of the queerness measures (e.g. number of same sex couples joint filing taxes) are positive
- Some zipcodes that are included in the data actually don’t have people living in them (like 20535 and 94128)
To avoid this issue, we first dropped any zipcodes with \(<100\) habitants, and then recalculated our outcome variable, based on the following formula:
\[gayborhood\_index = 40*ss\_tax\_index + 40*ss\_live\_index + 10*parade + 10*bars\] where
- \(ss\_tax\_index\) is a measure of what percent of couples filing jointly are same sex couples (normalized by the maximum value, on a 0 to 1 scale)
- \(ss\_live\_index\) is a measure of what percent of unmarried couples living together are same sex couples (normalized by the maximum value, on a 0 to 1 scale)
- \(parade\) is a boolean indicating if a pride parade passes through that zipcode (0 or 1)
- \(bars\) is a measure of how many gay bars there are in that area (normalized by the maximum value, 20, on a 0 to 1 scale)
This calculation accurately repurposes the original dataset for our use, which is not as concerned with splitting up men and women as was the original article Men are from Chelsea, Women are from Park Slope.
We then used a Yeo-Johnson transformation with \(\lambda = -0.268\) to transform our data and improve normality. This transformation was chosen as it allows us to keep our 0 values, which have an important physical representation in this dataset, without greatly compromising normality:
Variable Transformations and Selection
Additionally, we employed a LASSO model with penalty = 0.05, which is a “strict” penalty as it removed over 75% of our initially selected variables. Here is a list of selected variables which will appear in recipes to predict the value of our outcome variable:
| Name | train[, 2:11] |
| Number of rows | 1713 |
| Number of columns | 10 |
| _______________________ | |
| Column type frequency: | |
| numeric | 10 |
| ________________________ | |
| Group variables | None |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 |
|---|---|---|---|---|---|---|---|---|---|
| gayborhood_index | 0 | 1 | 1.09 | 0.50 | 0 | 0.77 | 1.08 | 1.41 | 2.6 |
| estimate_sex_and_age_total_population | 0 | 1 | 29591.70 | 19279.92 | 304 | 15253.00 | 26824.00 | 38830.00 | 114982.0 |
| estimate_sex_and_age_total_population_male | 0 | 1 | 14436.43 | 9404.58 | 194 | 7442.00 | 12986.00 | 18987.00 | 60595.0 |
| estimate_sex_and_age_total_population_female | 0 | 1 | 15155.27 | 9939.30 | 110 | 7846.00 | 13732.00 | 20095.00 | 58520.0 |
| estimate_sex_and_age_under_5_years | 0 | 1 | 1896.69 | 1502.35 | 0 | 846.00 | 1566.00 | 2521.00 | 12098.0 |
| estimate_sex_and_age_5_to_9_years | 0 | 1 | 1841.38 | 1431.04 | 0 | 825.00 | 1564.00 | 2478.00 | 10161.0 |
| estimate_sex_and_age_10_to_14_years | 0 | 1 | 1814.55 | 1407.43 | 0 | 812.00 | 1529.00 | 2414.00 | 9785.0 |
| estimate_sex_and_age_15_to_19_years | 0 | 1 | 1857.28 | 1497.53 | 0 | 785.00 | 1500.00 | 2449.00 | 10857.0 |
| estimate_sex_and_age_20_to_24_years | 0 | 1 | 2086.28 | 1829.05 | 0 | 850.00 | 1667.00 | 2801.00 | 18938.0 |
| estimate_sex_and_age_25_to_34_years | 0 | 1 | 4672.86 | 3638.25 | 9 | 2024.00 | 3941.00 | 6330.00 | 26926.0 |
A complete skim of the selected predictors is listed in the appendix. There are also a few variables which have missing values, which need to be imputed. We will use both KNN or Bagged Tree methods, and evaluate to see which method of imputation is more appropriate for this particular dataset. We expect that the flexibility of Bagged trees to be lend itself better to prediction, but note that KNN might be helpful for reducing runtimes.
Basic Recipe Steps
Our basic recipes will follow this formula:
Update the role of zipcode to be an identification variable, rather than a predictor
Dummy encode nominal predictors
Impute missing values with EITHER KNN OR Bagged Trees
Scale and center predictors around 0
remove predictors with near-zero variance (> 95% the same)
In the initial steps of model fitting, we plan to continue experimenting with dimensionality reduction after our initial LASSO model, especially with the addition of interaction terms followed by PCA in our penalized elastic net models.
Assessment Measures
For assessment, we continue our plan to use RMSE as a base metric, but will also include the Concordance Correlation Coefficient (CCC) to report the ability of our model to capture variance in our target variable.
Current Results
Currently, we have fit both a null model and a random forest model on our data. The results from both are below:
| Null Model | |
| mean_RMSE | Standard Error |
|---|---|
| 0.503 | 0.00206 |